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Abstract 


Connection Machine Lisp is a dialect of Common Lisp extended to allow a fine-grained, 
data-oriented style of parallel execution. This parallelism is organized around objects 
called zappings, which are similar to arrays or hash tables. Two syntactic constructs are 
introduced: the first allows existing Lisp functions to operate on elements of xappings in 
parallel, and the second provides a means of expressing general interprocessor communi- 
cation. Since a Connection Machine Lisp implementation may be embedded in an existing 
Common Lisp, it can be quite small, and benefits from the sophisticated programming 
environment of the underlying system. 

While the language was inspired by the architecture of the Connection Machine com- 
puter system, its design is relatively hardware-independent, and should be suitable for 
implementation on other parallel computers, such as the NYU Ultracomputer or NON- 
VON, as well as sequential machines. 


1 Introduction 


The development of the Connection Machine computer system [11,24] and other massively 
parallel computers has created the need for new programming languages in which massively 
parallel computations can be expressed. Existing languages and language extensions for 
vector and array processors are not entirely appropriate for these new architectures because 
they are based closely on particular hardware configurations, or because they lack a facility 
for general interprocessor communication. While the Connection Machine system has 
proven itself in the traditional supercomputing arena, on number-intensive tasks, it was 
also developed with symbolic computation and artificial intelligence (AI) in mind. 

Lisp has traditionally been the language of choice for AI researchers, and the ex- 
ploratory nature of such research has given rise to sophisticated programming environ- 
ments. Common Lisp [20] is a modern, general-purpose dialect of Lisp that has inherited 
much of the culture and spirit of its predecessors. In particular, many Common Lisp im- 
plementations provide integrated text editors, incremental compilation, and sophisticated 
debuggers. 

Connection Machine Lisp is a general-purpose language for data parallel computers, 
and is also intended to be a language for AI research and applications. We hope it will 
serve, to some degree, as a bridge between AI and supercomputing, and as a tool for 
both symbolic and numeric computation. (See [2] for a discussion of a Lisp compiler that 
produces high-quality numeric code for a sequential computer.) 

The language is described in the next section, and a number of programming exam- 
ples are given in section 3. In section 4, the language is compared and contrasted with 
other parallel and supercomputer programming languages. Our current implementation 
is described in section 5. 


2 The Language 


Connection Machine Lisp extends Common Lisp by adding a new data type, the xapping, 
and providing operations on this data type for parallel execution. The parallel operations 
include two primitive operators, a and #, and a collection of library functions. 


2.1 Xappings, Xets, and Xectors 


All parallelism in Connection Machine Lisp is organized around a data structure known 
as the zapping (pronounced “zapping,” and derived from “mapping”). Xappings are data 
objects similar in structure to arrays or hash tables, but they have one essential charac- 
teristic: operations on the entries of xappings may be performed in parallel. A xapping, 
like any other Lisp object, may be dynamically allocated, and its associated storage is 
automatically reclaimed when all references to it are deleted. Xappings may be of arbi- 
trary size (up to some reasonably high implementation-dependent limit), and may contain 
pointers to any other Lisp objects (including other xappings). 

A xapping is an unordered set of ordered pairs. The first item of each pair is called an 
indez, and the second item is called a value. Pairs are written as indez—> value, and all the 
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boy—blue girl—pink —green 
y & P g 


specifies that boy maps to blue, girl maps to pink, and all other objects map to green. 

The function xref (pronounced “ex-ref”, and similar to the Common Lisp function 
aref) returns the value of a xapping at a specified index. The value may be changed by 
using setf with xref. 


(setq x ’{boy—blue girl—pink —green}) 

(xref x ’girl) > pink 

(xref x ’martian) > green 

(setf (xref x ’girl) ’heliotrope) => heliotrope 
(xref x ’girl) => heliotrope 


2.2 Parallel Computation: a Syntax 


The function call mechanism of Connection Machine Lisp allows xappings of functions to 
be called as functions, provided that all arguments to a xapping of functions are themselves 
xappings. The result of such a function call is a xapping whose domain is the intersection 
of the domains of the function xapping and argument xappings, and whose range is made 
up of the results of applying each function to the values from the argument xappings at the 
corresponding indices. When a xapping of functions is called in this way, the individual 
function calls may be performed in parallel. Resynchronization occurs, at latest, when all 
of the parallel computations have completed. 
One could perform several additions in parallel by doing this: 


(funcall *>{—+} °[10 20 30 40] °[8 7 6 5 43 2]) 


The result of this call is a xapping whose domain consists of the integers 0 through 3 (i.e., 
the intersection of the infinite domain, the domain of a xector of length 4, and the domain 
of a xector of length 7), and whose range is formed by calling elements from the function 
xapping (always +) with corresponding elements from the argument xappings: 


[18 27 36 45] 


A special syntax using the alpha character, a, allows us to write parallel function calls 
more concisely than we did above. The expression az constructs a constant xapping with 
the value z. Using a, we can now rewrite the parallel function call shown above like this: 


(a+ ?[10 20 30 40] °[8 7 6 5 43 2]) 


Note that a xapping of xappings of functions may be called as a function, too, as long as 
its arguments are xappings of xappings, and so on. 


(aat+ *?[{i 2 3] [4 5 6] (7 8 9]] 
*C{9 8 7] [6 & 4] [3 2 1]]) 
=> [[10 10 10] [10 10 10] [10 10 10]] 
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however, there are a number of special forms and macros for which parallel execution is 
both meaningful and useful. 

Conditional execution is accomplished with aif, a parallel version of the if special 
form. To evaluate the expression (aif condition then else), we first evaluate the condi- 
tion, which must return a xapping. We then evaluate the then expression, but only at 
the indices for which the condition is true. We likewise evaluate the else expression at 
indices for which the condition is false. The result of aif is the union of the then and else 
xappings. For example: 


a(if (oddp -°[0 1234567 8 9]) ’odd ’even) 
=> [even odd even odd even odd even odd even odd] 


Note that both consequents of an aif are always evaluated, but at disjoint sets of indices 
(ie., in disjoint sets of processors). 

Local variable bindings may be established in each processor with alet, a parallel 
version of the let special form. Variables are bound to values specified in an initial value 
xapping for the duration of the alet body, in which each form is evaluated as if it were 
preceded by a. The result of alet is the result of the last form in the body. 


a(let ((x -°-f0 1234567 8 9])) 
(* x x x)) 
=> (0 1 8 27 64 125 216 343 512 729] 


The values at a set of indices in a xapping may be altered in parallel with asetf, a 
parallel version of the setf macro. (asetf old new) sets the value of each index appearing 
in both old and new to the value at that index in new. 


(setq x ?{a—-1 b—-2 c—3}) 
(asetf x ’{b35 c37 d—9}) 
x > {a1 b—5 c—7} 


2.3 Interprocessor Communication: 8 Syntax 


We can think of the a syntax as a way of broadcasting data and programs to different 
indices (i.e., processors). Another syntax, using the beta character, 3, is used to gather 
data and route it between processors. ; 

The simplest use of f is called reduction. The expression (ff z) takes a two-argument 
function f and a xapping z and returns the result of combining all the values of z using f. 
For example, 


(6+ ?>[0 1 23 4 65]) 


returns the sum of all the values in the xector, namely 15. Any two-argument combining 
function may be used, but the result is unpredictable if the function is not associative and 
commutative, because the order in which the values are combined is not predictable. 
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(scan #’max ’[1 6 2 7 3 4 2]) 
> [166777 7] 
(scan #’max ’[1 6 27 3 4 2] 
:segment ’[t nil nil nil t nil nil]) 
> [1667 3 4 4] 


A number of very simple functions are used idiomatically in reduction, combination, 
and scanning. These have been given canonical names to save typing: argi always returns 
its first argument, arg2 always returns its second argument, arb returns either argument 
unpredictably, and @ signals an error if called (it is usually used in combination when no 
collisions are expected). Uses of some of these can be seen in the examples below. 


3 Programming Examples 
3.1 Finding Prime Numbers 


On a massively parallel computer, we can compute succesive prime numbers in constant 
time by using the sieve method. We use two xappings: possible is t for indices that 
might be primes, and primes is t at each prime index we’ve found thus far. On each 
iteration we find the next prime (the lowest possible prime), record its primeness in 
primes, and eliminate multiples of that number from the possible primes. 


+33 Primes returns a xet of all the prime numbers 
333 less than N. 


(defun primes (n) 
(let* ((possible 
(make-xector n :initial-element t)) 
(primes 
(make-xector n :initial-element nil))) 
(asetf possible ’ [nil nil]) 
(do ((next-prime (position t possible) 
(position t possible))) 
((null next-prime) 
(aif primes (iota n) ’{})) 
(setf (xref primes next-prime) t) 
a(setf spossible 
(and spossible 
(not (zerop 
(mod «(iota n) 
next-prime)))))))) 


We use make-xector, which closely resembles the Common Lisp function make-array, 
to allocate the two xappings possible and primes. The possible xapping initially maps 
all indices to t, and the first asetf changes the possibility for 0 and 1 to nil. 
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symbol) of the object from which the links are directed. The is-a-segment xector con- 
tains t at each index that is the first in a group of links for a particular object, and nil 
in all other positions. The is-a-link xector specifies the first index in the group of links 
originating from the object to which the link is directed. 


is-a-value 
=> [snake lizard lizard reptile elephant ...] 
is-a-segment 


> [t t nil t t aead 
is-a-link 
> [3 3 6 13 6 send 


(iota (length is-a-value)) 
=> [0 1 2 3 a ited 


It may appear that our constraint that all links for a given object be adjacent would 
make adding links to existing objects an expensive operation, but this is not so, since 
shifting subranges of the xectors to make room for a new link may be done in constant 
time on a Connection Machine computer. 

Here is a function is-a-p, that returns t if object x has some path of is-a links to 
object y: 


(defun is-a-p (x y) 
(do* ((active a(eql eis-a-value x)) 
(old-active (make-xector (length active)))) 
((find y a(and eactive eis-a-value)) 
t) 
3; Save current active set in Old-Active: 
a(setf eold-active eactive) 
3; Spread activation to all links in group: 
a(setf eactive «(scan #’argi active 
:segment is-a-segment) ) 
;; Send activation to adjacent nodes: 
a(setf eactive 
°(Barb a(where (and eactive eis-a-link) 
eis-a-link) 
*{t})) 
;; If no more active than before, we’re done: 
(when (equal active old-active) 
(return nil)))) 


The active xector represents the set of active nodes, containing t for the index of every 
link we have traversed. We begin our search by initializing active to the set of links 
originating from x. If the origin of any active link is y—the object we are searching for— 
then we return t. If at the end of an iteration the active set is equal to the previous 
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xappings. (We have considered handling multidimensional arrays in Connection Machine 
Lisp by letting an index itself be a xapping: a 2 x 2 identity matrix would then be 
represented as 


{fo 0]-1 [0 1]-0 [1 0]-0 [1 1]-1} 


The difficulty here is that in this case we would like for two indices to be considered 
the same if they are equal rather than merely eql; however, the fact that xappings are 
mutable creates grave difficulties. What if the xector [1 0] used as an index in the identity 
matrix were mutated to be [1 1]? Remember that no two pairs of a xapping may have 
the same index. These nasty issues are the reason why indices are compared using eq1: 
the equivalence of indices must remain invariant under mutability of data.) 

NIAL [14,17]. Many of the comments about APL apply to NIAL, except that NIAL 
allows nested arrays. NIAL, like Connection Machine Lisp and unlike APL, allows user- 
defined functions to be used with the reduction and scan operators, and indeed all opera- 
tors. We think Connection Machine Lisp has a better notation (a and ¢) for nested uses of 
the apply-to-all construct (which NIAL calls EACH). Such nested uses do not occur so fre- 
quently in NIAL because apply-to-all is implicit in many NIAL operations; this is possible 
because NIAL has a different theory of data structures than Lisp [15,16]. In NIAL data is 
immutable but variables are mutable. (The remarks about NIAL apply for the most part 
to other “modern” APL implementations such as those of IBM, STSC, I. P. Sharp, etc.) 

FP [1]. Many of the ideas of Connection Machine Lisp, including the alpha notation, 
were borrowed from FP, and some of the aspects of FP programming style also carry over 
into Connection Machine Lisp. Like Lisp, however, Connection Machine Lisp is oriented 
around variables and less around functional composition; FP does not explicitly name the 
data to be operated on, but relies on combinator-like control of the flow of data. FP is an 
applicative language; data is immutable. 

QLAMBDA [5]. This is a dialect of Lisp that provides parallel control structures in 
the form of variant lambda expressions. Connection Machine Lisp organizes its parallelism 
around data structures rather than control structures, and thus may be more suited to a 
SIMD architecture than to a MIMD architecture; the opposite may be true of QLAMBDA. 

Multilisp [10,9]. Multilisp, like QLAMBDA, has parallelism organized around control 
structures rather than data structures. Multilisp introduces parallelism in two ways, one 
structured and the other extremely unstructured. The structured way allows the elements 
of a very particular data structure to be computed in parallel; this data structure is the 
list of arguments for a function call. (Such a parallel function call is introduced with the 
special keyword pcall1 to distinguish it from an ordinary function call.) The unstructured 
way is the use of future, which allows an arbitrary computation (the argument form to 
future) to proceed in parallel with another computation of arbitrarily unrelated structure 
(the remainder of whatever computation surrounds the execution of future, that is, the 
continuation). 

Symmetric Lisp [6,7]. There is a possible confusion between our notation and that of 
Gelernter, because his work also involves parallelism in Lisp and uses a notation involving 
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determines the meaning of the data field as follows: 

00 The data encodes a 32-bit signed integer. 

01 The data encodes a 32-bit IEEE floating point number. 

10 The data encodes some other Lisp object stored on the front end. 
11 This tag indicates a free block header (see below). 


The first 256 front end objects are reserved for extended-ASCII characters, so character 
objects may effectively be manipulated in the Connection Machine system itself. Other 
Lisp objects are assigned unique ID’s as they are stored in xecs. 

The xapping {a1 b—2 -—+3} might be stored as shown in Figure 1. 


5.1.2 The Rendezvous Mechanism 


When a parallel computation is to be carried out on elements of xappings with different 
domains, values with the same indices must be brought into the same processors. This 
operation is called a rendezvous, and is facilitated by the following convention. Xecs 
that serve as domains of xappings hold not the xapping indices themselves but rather 
the addresses of the processors that serve as rendezvous points for those indices. Each 
processor may serve as the rendezvous point for one Lisp object, and stores that object in 
its own memory. 

Suppose we want to add the elements of two xappings, {foo+10 bar—20 baz—30} 
and {bar2 baz—3 rag-—4}, stored as indicated in Figure 2. Let us say that the ren- 
dezvous point for foo is processor 0, bar is processor 1, baz is processor 2, and rag is 
processor 7. The values of the first xapping are sent to a temporary location addr1 at the 
rendezvous points for their corresponding indices, so 10 goes to processor 0, 20 goes to 
processor 1, and 30 goes to processor 2. Next we send the values of the second xapping 
to temporary location addr2 at their rendezvous points: 2 goes to processor 1, 3 goes to 
processor 2, and 4 goes to processor 7. The processors that receive values from both xap- 
pings form the intersection of the domains of the arguments, and carry out the addition, 
leaving the result in location addr3. 

Rendezvous locations are calculated as data is sent from the front end to the Connec- 
tion Machine processors. Since each processor “knows” the Lisp object for which it is the 
rendezvous point, finding it is simply a matter of broadcasting a request for that processor 
to identify itself. Only the combining form of 6, in which two xappings are supplied, may 
require new rendezvous points to be assigned to values already stored in the Connection 
Machine memory. 


5.1.3 Storage Allocation 


The major portion of memory within each Connection Machine processor is reserved for 
heap allocation of xecs. When the system is initialized, the heap memory is divided into 


85 The Connection Machine Implementation 17 


rendezvous-id 


Figure 2: A rendezvous. 


34-bit “stripes” where xec’s cells will be allocated, and the head of each stripe, in processor 
0, is filled with a free block header whose data bits encode the number of free elements 
following that header. In a system containing 65,536 processors, each stripe would be 
initialized with a header indicating 65,536 free xec cells. 

To allocate storage for a xec of length n, we choose a stripe and broadcast a request for 
all free block headers in the given stripe who head blocks of at least n elements to identify 
themselves. We choose one of these free headers as the first element in our new xec, and 
initialize the xec structure accordingly. The old free block header must be modified so 
that the same storage isn’t allocated twice, and any free elements following the new xec’s 
elements must be headed with a new free block header. If no free block of the required 
size is found in one stripe, the process is repeated for the following stripe. In the worst 
case, this allocation algorithm runs in time proportional to the number of stripes; however, 
in practice the technique of starting with the stripe following the last stripe used almost 
always results in success on the first attempt. 

Reclaiming the storage associated with a xec requires us to track the xec structure 
during front end garbage collection. Different front ends use different garbage collection 
techniques, so the details of how this is done can vary greatly, but somehow we derive the 
set of “live” xecs. By marking the cells associated with these xecs and complementing 
this set, we can construct the set of free xec cells, and in turn construct new free block 
headers. 


5.2 The Compiler 


The compiler translates a Connection Machine Lisp program into a Common Lisp program 
containing calls to runtime system functions and PARIS operations. This program is then 
compiled by the front end’s Common Lisp compiler. The translation process can be broken 
down into three phases: annotation, optimization, and code generation. These phases are 
not distinct passes over source and intermediate code in the traditional sense, but are 
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defined using defoptimize: 


(defoptimize iota ((length) 
for-value storage-class) 
(declare (ignore for-value)) 
‘(the (xector fixnum ,@(when (constantp length) 
*(, length) )) 
(iota* ,storage-class ,length))) 


Here, a call to the user-level function iota is transformed into a call to an internal function, 
iota*, which accepts a storage class argument. Since we know that all calls to iota return 
a xector of fixnums, we wrap a declaration of this type around the optimized code. If the 


length argument is constant, we can specify the length component of the type specifier as 
well, 


5.2.3 The Code Generator 


The code generation phase uses the type information from the annotation phase to further 
transform the optimized program. It is at this point that “parallel” calls to Lisp functions 
are turned into calls to PARIS routines. This database is built in a straightforward manner, 
using the macro defcmop (“DEFine CM OPeration”). 

This macro takes a Lisp function name and information about how to execute that 
function in parallel (which may vary depending on the number and types of its arguments) 
and how to reduce and combine using that function. The following example, which supplies 
information about the + function, demonstrates the sort of information one may store in 
the database. 


(defemop (+ :alu) 

identity 0 

:unary cm:move 

:binary (:fixnum cm:+ 
:flonum cm: f+) 

in-ary t 

:reducer (:fixnum cm:global-add 
:flonum cm: global-float-add) 

:combiner (:fixnum cm:send-with-add) ) 


The first parameter specifies the name of the Lisp function being specified its general 
class; in this case, the + function is declared to be a simple arithmetic or logical function. 
Specifics about this operation are provided with the keyword arguments: 


:identity indicates that the result of a zero-argument call to + is 0. 


sunary specifies that a one-argument call to + is simply a move from the argument’s range 
xec into the result’s range xec. 
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by making a relatively small number of changes to an existing implementaton of Common 
Lisp. 

The language allows one to program operations on large quantities of data. The 
xapping data structure may be interpreted operationally as describing a collection of values 
conceptually stored one per processor, where processors are labelled by Lisp objects (such’ 
as symbols). One may write a given piece of code from either a local perspective, focusing 
on the operations applied to a single data element, or from a global perspective, focusing 
on arrays and transformations upon them as a whole. 

While the design of Connection Machine Lisp was inspired by the architecture of the 
Connection Machine computer system, it relies on no unusual features of that computer 
system other than providing a large number of processors and communication among 
them, and should be suitable for implementation on other parallel computers, such as the 
NYU Ultracomputer [18] or NON-VON [19], as well as sequential machines. 
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